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Abstract 

The possibility of an anomalous coupling between the top and 
charm quarks and the gluon field is explored in a model-independent 
way using an effective Lagrangian that is gauge-invariant under a non- 
linear realization of SU(3)c>< SU(2)iX U(l)y. Even for the current 
200 of integrated luminosity at the Tevatron, the new physics 
scale that strongly modifies the coupling of t-c-g must be larger than 
about 2.5 TeV if no signal is found within a 3a confidence limit. For 
1 fb -1 of data, this constraint can be pushed up to 3.8 TeV. 
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1 Introduction 



With the discovery of the top quark at Fermilab by the D0 and CDF 
collaborations, it has become natural to study its interactions with the gauge 
bosons. The Standard Model (SM) completely predicts how the top quark 
should behave under these interactions, so any deviation from this behavior 
would provide us with a probe of new physics beyond the SM. 

The top (t) quark is very heavy, about 35 times that of the next heaviest 
quark, the top's weak partner, the bottom (b). For this reason it is a likely 
place to search for new physics. If new physics is found in the top quark 
sector, it is possible that this new physics could explain why the top is so 
heavy and how its mass is generated. This could in turn provide us with 
clues as to how the other quark masses arise (a question the SM makes 
no effort to address). Perhaps there is new physics specific to the third 
family, physics which can explain why the top, bottom, and tau lepton are 
so much heavier than their first and second family counter-parts. Or there 
could be new interactions that are not really involved in producing the large 
masses, but coupling more significantly to particles with large mass, and thus 
can be detected by studying the top, while only affecting the other quarks 
insignificantly. 

The top mass is of the order of the Electro- Weak Symmetry Breaking 
(EWSB) scale v = 246 GeV, and thus provides a probe of the physics as- 
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sociated with the generation of the masses for the weak gauge bosons. The 
Higgs mechanism of the SM requires a neutral scalar particle (the Higgs bo- 
son) which has yet to be directly detected experimentally, but could affect 
(although marginally) low energy experimental results through loop effects. 
If the Higgs mechanism with Yukawa interactions is responsible for the gen- 
eration of the fermion masses, the Higgs boson should have a coupling with 
the top quark of the order of m t /v, and thus interactions involving the top 
quark may provide a probe of the Higgs physics. 

The SM does not contain tree-level flavor-changing neutral currents (FCNC), 
though they can occur at higher order through radiative corrections. Because 
of the loop suppression, these SM effects will be small, and so large FCNC 
provide a window into physics beyond the SM. In this paper, we are specif- 
ically interested in the possibility of a top-charm-gluon (t-c-g) coupling. As 
we have explained above, it is natural to look to the top quark as a win- 
dow to new physics. If this new physics may participate in the generation 
of fermion masses, it is reasonable to assume that the coupling of the top 
to the other up-type quarks should be proportional to v /m t m c ( u ) 0. This 
leads to the conclusion that the top-charm coupling is more likely to lead to 
measurable effects than the top-up coupling. A study similar to ours of the 
t-c-Z anomalous coupling can be found in |2|. §. 

There are a number of interesting models {|, [5| in the literature that can 
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produce this kind of anomalous coupling, and for this reason it is useful to 
study this kind of interaction experimentally in order to constrain parameters 
in these theories. In this paper we study the t-c-g vertex in an effective 
Lagrangian, model-independent way. We show how it is possible to use 
single-top production data from the Tevatron to constrain the t-c-g coupling. 

2 Theory 

To incorporate new physics, we consider an effective Lagrangian, 

^eff — A) + Ci, (1) 

where L\ contains operators of dimension higher than four, multiplied by 
coefficients with appropriate dimensions of mass to insure that the dimension 
of the Lagrangian as a whole remains four. It is including terms of this kind 
that leads us to call this an "effective Lagrangian"; since the resultant theory 
is not valid to an arbitrarily high energy scale, it is not a fundamental physical 
theory. Instead, it represents the theory that is "effective" at a lower energy 
scale where the energy is too low to allow us to see the full details of the 
underlying physics. The coefficients with dimensions of mass in front of the 
effective terms characterize the mass scale at which new physics must enter 
the theory if any non-SM effect is to be found. In our case, since we wish to 
consider the possibility of a flavor- changing gluonic current, Co will be the 
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QCD Lagrangian, 

Co = -\G%G^ + q i^D^q - m g qq- (2) 

where = — ig s ^G a ^ and G aiiV is the usual gluon field strength tensor. 

We are interested in exploring the possibility that the gluon current can 
couple the top and charm quarks at tree level in an effective theory valid up 
to a scale of A. C\ must be constructed in such a way as to accomplish this, 
respecting the SU(3)c7 gauge invariance of QCD. Our effective theory must 
contain a cut-off mass scale, A , that is appreciably larger than the energy 
scale at which we do calculations, for as stated above, it is only in this region 
that the effective theory is valid. Since any higher dimension operators we 
introduce will be suppressed by a power of the cut-off mass that fixes the 
over-all dimension of the term at four, we expect that the effective operators 
of lower dimension should be more important than the higher dimension 
ones. The lowest dimension effective operator we can add to produce a t-c-g 
coupling is dimension five. It is given by: 

where A represents a mass cut-off scale above which our effective theory 
breaks down, and kl(r) are dimensionless parameters that relate the strength 
of the "new coupling" to the strong coupling constant g s for left (right-- 
handed top quarks. If the imaginary part of klk* r does not equal zero, this 
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interaction will violate CP conservation. For simplicity, we will restrict our 
study to the case where kl(r) is real, so CP is conserved. 

L\ is invariant under local SU(3)c gauge transformations. It is also invari- 
ant under a nonlinear realization of the broken symmetry SXJ(2)lX U(l)y — > 
U(l)£Af, as in the Chiral Lagrangian 0, || |j. An example of this is provided 
in || where the SU(2)lX U(l)y symmetry is realized in such a way that the 
fermion fields transform as their XJ(1) em charges. It is clear that L\ is in- 
variant under this realization of SU(2)iX U(l)y because the top and charm 
have the same electric charge and the gluon field is an Electro- Weak singlet. 

Once the terms in the Lagrangian are specified, one can derive the Feyn- 
man rules corresponding to the vertices in the theory. The form of the gluon 
field strength tensor will produce two different types of vertices (for each 
K L{R)) ~ a four point coupling of two gluons and the top and charm quarks, 
and a three point vertex coupling of one gluon line and the top and charm 
quarksQ 

In order to study the t-c-g couplings introduced by C\ and to determine 
the minimum energy scale A that would contribute to these couplings, we 
consider the production of top-charm at a hadron collider. There are seven 
tree level diagrams (for each top quark helicity) that contribute to tc, pro- 
duction, as shown in Figure 1. As we will argue below in Sec. 3, it is useful 

1 In Appendix A we tabulate the Feynman diagrams resulting from L\ and QCD, in- 
cluding the relevant vertex factors. 
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to require that the invariant mass M t c > 300 GeV to separate the signal from 
the background. By energy conservation, this is the same as requiring the 
invariant mass of the incoming partons yl > 300 GeV. As a result, the qq 
annihilation diagram dominates the diagrams that have gluons as incoming 
partons. This is because the parton distribution functions^ are such that 
for large Vs, most of the momentum is carried by the quarks; the proba- 
bility of observing a gluon carrying a momentum fraction in the range we 
are interested in (x = y/I/^/s > 0.15) is much smaller than that of a quark 
carrying the same momentum. The gg luminosity is about 0.20 times the 
qq luminosity. Therefore, for > 300 GeV, we can approximate the whole 
process by the single qq annihilation diagram. 

Considering for the moment just the tc production without any decays, 
we find that the constituent cross section after integrating over the final state 
phase space is given by 

m - tc) = 9 ' { ;l + / R) (l ~ f' 2 (* + 2m?). (4) 

Note that in the high energy limit (i.e. for large y/§), o is a constant and 

does not have the usual functional form of 1/s. For simplicity, we do not 

study Kum separately in this work; here we are interested in the energy scale 

at which new physics largely modifies the t-c-g coupling. In the rest of this 
2 The CTEQ2 leading order fit parton distribution functions are used in all calculations. 



7 



paper, we restrict ourselves to fti = % (a vector-like coupling)^. This leads 
us to define, 

^ 2 = 4 + 4- (5) 

The constituent cross section contains k and A only in the combination k/A, 
thus it is convenient to define, 

A = ±, (6) 

so that k has been absorbed. This leaves us with one parameter to compare 
with experimental data. For \k\ ~ 0(1), A is about the same as the cut-off 
scale described above, and characterizes the effective strength of the new 
couplings. 

One could also try to learn the theory's parameters by considering the 
decay t — > eg, but this is not very well suited to a hadron collider because 
of potentially large backgrounds from which the signal events cannot easily 
be distinguished. However, for completeness, we present the decay width 
t — > eg, in the rest frame of the top quark, 

r( t -c ff) = ^. (7) 

The dependence on m t of this result may easily be understood from dimen- 
sional arguments. Because the t-c-g coupling contains the factor A" 1 , the 

3 In principle, one could look at the angular distribution of the top quark's decay prod- 
ucts to determine the coefficients kl and kr for a given A. 
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width must be proportional to A -2 . If the charm quark mass is taken to be 
zero, the only energy scale is m t and since the width must have the dimension 
of energy, it must be proportional to m|. In order to use this information to 
determine A, we find the ratio defined as, 

T(t - eg) 



R 



teg 



where, 



r(t w + b) 



T(t -> W+b) 



1 
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mf _ 




mf _ 



(8) 



(9) 



y/2 16tt 

is the decay width T(t — > W + b) JT^|. Combining these two equations to find 
Rtcg, we find, 



R, 



y/2QAa a 



teg 
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h 2^k 










m t _ 



(10) 



3A 2 G F 

We will discuss the implications of this result in Sec. 5. 

3 Signal and Background 



In this section we discuss the detection efficiency of the signal events at 
the upgraded Tevatron (a 2 TeV pp collider). The top quark decay mode 
studied is t — > bW + (— > t+v) for t = e or \i. 

Including the decay of the top quark, the process qq — ► tc results in the 
final state Wbc. Thus, the signature of this process is an energetic charged 
lepton, missing Et, a 6-quark jet from the top decay, and a light jet. This 
signal has (neglecting all of the quark masses except for that of the top) 
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kinematics similar to the Standard Model process qq — > W* — > £&, which 



has been studied in Ref. [11, 12|, including the relevant backgrounds. In this 



study we consider the following intrinsic background processes: 

• tq produced by VT-gluon fusion 

• Wbb 

• t6 -f Wbb 

• ft -> W'W+bb 

• Wjj 

The potentially large background from Wjj can be reduced by requiring a 6- 
quark to be present in the final state (i.e. 6-tagging). The CDF collaboration 
has effectively implemented this procedure by using a silicon vertex detector 
(SVX). For Run II, the efficiency is estimated to be 60% per 6-jet {jP T > 20 
GeV and ^ within the SVX coverage), with a probability of less than 1% 
for a light quark or gluon jet to be mis- identified as a 6-jet 0. We require 
that only a single 6-quark be identified in the final state in order to accept 
the event, and we have ignored the possibility of a c-quark being mis-tagged 
as a b- quark. 

The number of events for the signal and backgrounds mentioned above 

were calculated for the TevatronQ (assuming an integrated luminosity L = 
4 Vi = 2 TeV pp collider 
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1 fb l ) using the Monte Carlo program ONETOP [|T2| , [T3|1 , except for Wjj 



which was calculated using PAPAGENO ||14|| . The W-gluon fusion rate is 
calculated from the two-body process, and normalized by the total rate as 
explained in J12|| . The top mass is taken to be m t = 175 GeV and the masses 
of the lighter quarks are neglected. The signal and background cross sections 
include the decay W — > tv (where I — e, /V) and both the W + and W~ 
production modes are included. (Since the dominant W decay modes are 
hadronic, ignoring the possibility of W — > hadrons reduces the signal, but 
the high px charged lepton resulting from the leptonic decay modes provides 
an excellent trigger at a hadron collider.) For now, we choose A = 2 TeV. We 
will discuss how to determine this parameter from experimental data below. 

Table 1 lists the primary cuts imposed to simulate the detector accep- 
tance, pt denotes transverse momentum, rj denotes the pseudo-rapidity, and 



ARji = y' A(pj £ + Arjj h where Acfiji is the difference in the azimuthal angle 
between j and i. The tc production rate is not very sensitive to the pseudo- 
rapidity cuts because the decay of the heavy top quark will generally produce 
6-jets and leptons in the central region. Thus, narrowing the pseudo-rapidity 
range should not significantly affect the signal rates, but will reduce the Wbb 
background. Only minimal px cuts for the jets are imposed here. In our 
final analysis these cuts have been chosen specifically to enhance the signal 
versus the background, and will be explained below. The events remaining 
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after these cuts (not including the 60% 6-tagging efficiency) are shown in the 
second column of Table 2. 

From the form of the constituent cross section given in equation (4), we 
see that for large s, the cross section for tc production does not fall off as 1/s 
as it will for most of the background processes. This suggests that one way 
to improve the signal to background ratio is to impose a cut on s, which by 
energy conservation is equivalent to imposing a cut on M t c- We find that the 
best result is obtained by requiring M m > 300 GeV. This presents something 
of a problem because there is no way to directly measure M t c- In order to 
reconstruct M t z we can use the lepton momentum and the missing transverse 
momentum in the equation, 

M^ = (p e +p u ) 2 , (11) 

to find two solutions for the missing component (i.e. the component along 
the z-axis) of p u , where p v is the neutrino 4- momentum. The solution which 
better fits the decay of an on-shell top quark, 

M? = (p e +p u +p b ) 2 , (12) 

is chosen^. The other solution is discarded. For the signal, there is only 

one 6-jet, so we can do this without any problem. However, for most of the 

5 In our calculation, the width of the top quark is included via the Breit-Wigner pre- 
scription. For a 175 GeV top quark, the SM width is about 1.5 GeV. 
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background processes, there is more than one 6-jet, and it is impossible to 
know a priori which one should be associated with the top decay. To handle 
this, we randomly decide which 6-jet to use for those processes which contain 
more than one 6-jet in the final state. In order to allow for an off-shell W, we 
iterate this procedure three times, generating a different Breit-Wigner mass 
for the W each time, and keep the first result in which a solution exists. 
Once we have determined the four- momenta p u , pt and p c it is simple to 
use energy conservation to reconstruct the invariant mass M w . We impose 
the cut Mtc > 300 GeV on the signal and backgrounds, and the results are 
displayed in the third column of Table 2. 

The tt background is reduced by rejecting events that contain evidence of 
the decay of an additional W boson. Specifically, events containing an addi- 
tional lepton with p l T > 20 GeV and \rje\ < 2.5, or additional distinguishable 
jets with p 3 T > 20 GeV and \rjj\ < 3.5 are rejected (two parton jets are con- 
sidered distinguishable if ARjj > 0.4). This results in a drastic reduction of 
the tt background, leaving a very small number of events from the dileptonic 
decay mode of tt. The effects of these cuts have been taken into account in 
the second column of Table 20. 

In order to suppress the background from Wbb, we can make use of the 

fact that a 6-quark produced from a top decay can be expected to have a 
6 To this order, they do not have any effect on the other processes listed in Table 2. 
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large pr due to the heavy top mass (a typical value is about one third of the 
top quark mass). The bb jets from the Wbb background are produced by the 
decay of a virtual gluon, and should have a momentum distribution that is 
more evenly distributed. We found that the cut p b T > 30 GeV reduces the 
background and signal in such a way as to provide the best enhancement of 
the significance of the signal. The effects of this cut can be seen in the fourth 
column of Table 2. 

It is also desirable to impose a cut on p^. Because the signal process 
occurs in the s-channel, and the W-gluon fusion background is t-channel, the 
distribution of the pt of the signal c-jet should peak at a higher value than 
the light jet from W^-gluon fusion. Adjusting the cut to find the value that 
provides the best enhancement of the significance of the signal leads us to 
p c T > 30 GeV. As can be seen in the fifth column of Table 2, this cut has 
virtually no effect on the signal, while slightly reducing the W-gluon fusion 
and Wbb backgrounds. 

4 Determination of A 

In order to set bounds on the parameter A, it is necessary to look for 
evidence of a signal distinguishable from background fluctuations. We require 
a 3cr effect as our criterion for judging the signal to be distinguishable from a 
background fluctuation, that is, we require the probability for the background 
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to fluctuate up to the observed level to be less than 0.27%. The number of 
signal events, N s , has a simple dependence on the integrated luminosity and 
A, 

Ns = £ (13) 

The coefficient a, which characterizes the acceptance of signal events mul- 
tiplied by the signal cross section after extracting the factor 1/A 2 , is deter- 
mined by our Monte Carlo study at A = 2 TeV and L = 1 fb^ 1 to be a = 
616 TeV 2 /6 _1 . The number of background events, Nb, is just the integrated 
luminosity multiplied by the total background cross sections. Once Nb has 
been determined for a given L, we can determine the Ns that will provide a 
3a effect using Gaussian statistics (since N B and N s > 10). From N s one 
can then determine the minimum A such that a 3a effect is not observed at 
the integrated luminosity of interest. In Table 3 we list the constraints on 
A min , the minimum value of A for which new physics will show up in t-c-g 
couplings, for several different integrated luminosities provided a 3a effect is 
not observed at the Tevatron. 

We expect that because we are examining the high invariant mass M m 
region, the theoretical uncertainties due to the choice of the scale, Q 2 , in 
as, and in the parton distribution functions used for calculations, should 
be small (Note that in the relatively large x region, the parton distribution 
functions are well determined by deep inelastic scattering data.) In order to 
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quantify this effect we have examined the signal cross section using Q 2 = y§ 
and Q 2 = \Jm 2 + p l T 2 . We find that the cross sections in these two cases 
differ by about 17%, which can affect our determined A by about 10%. 

5 Discussion and Conclusions 

We have shown that it is possible to introduce an operator of dimen- 
sion five into the QCD Lagrangian that couples the top and charm quarks 
to the gluon field. This operator can be constructed to respect the SU(3)c 
local gauge symmetry of QCD, and the broken symmetry SU(2) £ xU(l)y — ► 
U(1)em realized non-linearly, as in the Chiral Lagrangian. Because the op- 
erator is dimension five, it is divided by a parameter with the dimension of 
mass, A, that characterizes the energy scale at which new physics will modify 
the t-c-g interactions. 

We have completed a Monte Carlo study of the process qq — > tc and 
the SM backgrounds present at the Tevatron to show how to enhance the 
ratio of the signal to the background to improve the limits one may set on 
the minimum value of A, A min . As shown in Table 3, our results indicate 
that even with an integrated luminosity of 200 pb" 1 , the lack of a 3a signal 
at the Tevatron requires A > 2.5 TeV (assuming a 6-tagging efficiency of 

60%[] ). Assuming A = 2.5 TeV, there would be 19 signal events and 41 

7 A 30% 6-tagging efficiency will reduce the constraint on A min to about A min > 1.7 
TeV. 
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background events produced at the Tevatron with an integrated luminosity 
of 200 pb^ 1 . A higher integrated luminosity allows one to set even stronger 
limits on A. For instance, for 1 fb^ 1 of data, A > 3.8 TeV if no signal is 
found at a 3a level. If a signal is found, we expect at least 43 signal events 
with 207 background events. From equation (10) we see that if A = 2.7 
TeV, the branching ratio t — > eg should be about 13%. This suggests that a 
detailed study of the decay of the top quark can probe A, but because of large 
backgrounds from which this signal cannot easily be distinguished, study of 
tc production should provide a better way to set experimental bounds on A. 
Thus, a detailed study of single-top production at the Tevatron is useful in 
that it will allow one to set bounds on the possibility of this type of anomalous 
top quark coupling. 

In this work, we did not consider the other possible single-top production 
processes qc — > qt ,qc — > tq , and gc — > gt as shown in Figure 2. We speculate 
that due to the large backgrounds discussed in Sec. 3, it is necessary to 
impose a large pr cut on the final state parton jet q, q, or g. This cut should 
significantly reduce these contributions to the signal, making it difficult to 
observe. Further study is needed to better understand this. 
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Table 1: Fundamental Kinematic Cuts 



p J T > 10 GeV 
Vt > 20 GeV 
M < 2.5 
\Vj\ < 3-5 
\Vb\ < 2.0 
AR je > 0.4 
ARjj > 0.4 
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Table 2: Total number of events at the Tevatron, including 
kinematic cuts, for L = 1 /6 1 and A = 2 TeV. The "Fundamental 
Cuts" denote those in Table 1. 



Process 


Fundamental 


Mtc > 300 GeV 


V b T > 30 GeV 


V C T > 30 GeV 


6-tagging 


tc 


308 


276 


256 


256 


154 


W-g fusion 


188 


168 


152 


132 


79 


Wbb 


168 


124 


88 


72 


35 


tb 


40 


36 


28 


28 


13 


tt 


4 


4 











Wjj 








4000 


80 


Tot. Background 








4232 


207 
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Table 3 : A min for several Integrated Luminosities 



L 




N s 


N B 


200 pb~ l 


2.5 TeV 


19 


41 


1 fb- 1 


3.8 TeV 


43 


207 


2 fb' 1 


4.5 TeV 


61 


414 


10 fb- 1 


6.7 TeV 


136 


2070 


20 fb' 1 


8.0 TeV 


193 


4140 
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Figure 1 : Feynman Diagrams for Single-Top 
Production Via t-c-g Vertices 
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Fig la: qq annihilation 




t, c 




t 



c 



Figure 2 : Representative Feynman Diagrams for Top 

+ Light Jet Production 



t c 



V ► 




t 



Q 



Q 



Appendix A: Feynman Rules for QCD and t-c-g operator 
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t-c-g Vertices 
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